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1. INTRODUCTION 
In this paper, the well-known large-scale unconstrained minimization method has been considered: 


min f(x), x e R” (1) 


Where f: R” — R is continuously differentiable and the matrix of the first partial derivative g(x) = Vf (x) is 
available. x € R”, n is a dimensional of the vector x, the CG algorithm is among the most efficient 
optimization algorithms for getting the minimum of the function (1) especially for large-scale problem [1]. 
Nevertheless, the CG algorithm is one of the more excepted choices in the big scale problem solving, as this 
method does not need any matrices [2]. The behaviour of the unconstrained optimization problem (1) is onset 
with a starting guess x) E R”, then the CG algorithm would output a sequence of points {x;}/29 using the 
repeated form which is denoted in the next equation: 


Xi+1 = Xi + Witi (2) 


where wj; the length of a step is calculated with a suitable line search method and t; is the direction of search, 
that is getting as follows: 


—Git1 fori=0 


lisa = (3) 
—Gisi + Piti foriz1 
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where g; is the gradient vector of the function f (x) and B; is a small value used to correct the path of search 
at x; . There are a number of well-known conjugation method formulas given by see [3]-[7]: 


FR _ |lgisall? j 

: ligili? (9 
T 

PRP _ Ihi 

pS 2 (5) 
llgill 
F 

HS _ Ji+1 Yi 

aay co (6) 
ti Yi 

DY _ llgissll? 

ie FF alin 7) 
ti Yi 


where yi = gi+1 — gi is the difference gradient of the function f(x) at the points xi+1, Xi respectively, and 
more details for the coefficient B; can be seen in [8]-[12]. 

These aforementioned methods have been studied by many researches including [13], [14], most of 
these methods studied the features of the conjugate gradient approach, recently there are many attempts to 
discover a recent formula for conjugate gradient methods which have good numerical execution and 
satisfying a global property and that is the same aim of our research, to establish this convergence property it 
is required to compute the step w; > 0 with some conditions such as week wolfe condition (WWce) [15]: 


f (xj + witi) — f (xi) < piwi gi ti (8) 

ggi ti < g(x + witi) ti < -p2 gi ti (9) 
or by using strong wolfe condition (SWc) which satisfy (8): 

|g Cxi + witi) til < -p2 gi ti (10) 


where 0 < pı <§, pz = 0. There are many other formulas that have been proposed by various scholars, for 
more details see [16]-[20]. The search direction is also important to determine the amount of the function that 
is ensuring the reduction of the search direction therefore we use quasi Newton method: 


t= — G'g (11) 


where G; is a matrix which is asymmetric and non-singular of the accession of the Hessian matrix which is denoted 
as a matrix of identity in the first step. The structure of this article is sequential as: first, a recent formula for the 
coefficient £f is derived, while the sufficient descent property and global convergence is presented in the next 
section, after that, the numerical facts results are presented. Finally, the conclusion is presented in the last section. 


2. NEW FORMULA OF £; 

The self-scaling quasi-Newton will be utilized to scale the Hessian matrix G;, [21], [22] scale some 
terms of broyden fletcher glodfarb shanno (BFGS). Our technical method is to scale all the terms of BFGS by 
multiplying G; by a scalar ĝ, then the direction becomes: 

ti+1 = ~Ê Gi+1i+1 (12) 


where ĉis a self-scaling factor and there are several types of the scalar ê such as [23], [24]: 


= zx (AlBayati) (13) 
a yisi ; 

E= tag (AlBayati & Maha) (14) 
ê = a Uf (a) = f+) + 57 Iira] -2 (Biggs) (15) 


Now the (BFGS) formula can be written in the form: 
T 


= Givis} +siy] Gi sis? | yT GiYi Sis? 
Gis = Gi -( Tess + T+ ek ee (16) 
Si Yi Si Yi SiYi SiYi 
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To scale the Hessian matrix G we have to use the self-scaling quasi-Newton method, by multiplying (16) by 
ê then: 


asl t+siy! G; st 
Gy = eG: = (Hen “1) ELL Si ist 4 Vii yp GiYi Sisi ze] (17) 


T 
Si Vi st yi st yi ‘siyi 


Now if we choose ê from (15), and replace G; by I then (17) will be refer to memory less (BFGS) and written as: 


T T T 
A YiSi +SiYj sist yT Yi Sisi 
Giga = ê |I — (ERRi) 4 Si p iyi Sisi (18) 
Si Yi sim s{yi sf 


In order to compute the new parameter £;, both sides of (18) will be multiplied by g;,,, from 


ti+1 = —Ê Gi41 i+, and from (3): 


T T T. T 
A lIis Yi Ji+1 Si Gita Yi Yi Si Gita 
—gi+1 + Biti = è (- +EP y p TE g, Ahs — ns SLs «,) (19) 
Gi+1 Pi $ Ji+1 sly; Vi syi l syi l syi sii i 


now if we use é from biggs (15) and for simplicity, we assume the term: 
F = fi — fia! Si Jisa (20) 


to simplify the steps of derivation, we have: 


T T T T 
soir 6 Si Ji+1YitYi Ji+1Si Si Ji+1YitYi Ji+1Si 
+ Bt; F] + esa [F a = 
—Gi+1 Pit l sT yi [ ] Ji+ı sTyi [ ] sly; sTyi 
T T T T T 
6 [F] Si Ji+iSi 4 9 sf Gi415i 6 PF) %i%i SLGit1Si | Yii SiGin1 Si 21 
Tey E eee mela e a Bee e aT AT ( ) 
Si Yi Si Yi Si Yi Si Yi Si Yi Si Yi Si Yi Si Yi 


now by multiplying (21) by y7 and divide both side of (21) by yf t;: 


T T T T T T T T T 
^ Y Ji+ı aii 6Yi Gita yi. Gita 6 Si Gita Vi Vit Vi Ji+1Yi Si Si Ji+1-Yi Vit Vi Ji+1-YVi Si 
E SOBA ppc DES EE SE eh E Be dwt SE BAIS 


yi iti sf yiyi ti yleu yi tis yi sf yiyi ti 
T T T T T T T T 
6 [F] sf Ji+1Yi Si 2 Si Tin, Si_ 6 [F] Yi Yi SiJi+1Yi Si 2 Yi Yi _ Si Ji+1Yi Si 22 
Te: yles Ta Tey le Ta. TE Tae Tae ye T.. (22) 
Si Yi ViPS] Yi Si ViVi ti Si Yi Si Yi SiYiYi ti Si YiYi ti Si Yi 
Since s? = wt? 
T T T T T T 
B: yi ee 6Yi Ji+1 [F] +2 Yi Ji+1 6[F] wti Ji+1Vi YitYi Ji+1:WVi ti _ 
; yt wth yy} ti Tti wtp ys eee 
T T T T T T T T T T: 
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T T 
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wth yiyi ti 
T T T T T T tT T 
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Now subs (F) of (20) in (25) we get: 
YT Gi+4 A Gita P gi+a 
= Hgin Ser 16( + ST giys)] + 2o ES 
ty; Tai tly fi— fira Ji+1 Eo 
T T T 
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And that is the final form of our new coefficient. If the direction is exact line search, then B; will be 
reduced to Hesten Stefile method. However, if we used inexact line search with Wolfe type line search then 
our algorithm of a new method is as the following: 


Algorithm (1) 

Given x)€R" and w,;>0, set i=1. 

Stepl: set t;=-g,=-Vf(x%) , if llgill<e , then go to end. 

Step2: determine w;>0 satisfying Wolfe type line search in (8) and (9). 

Step3: calculate a new iteration (Xi) by (2) and gis, -if |lgiml| <€ then go to end, 
else go to step (2). 


3. THE SUFFICIENT DESCENT AND GLOBAL CONVERGENCE PROPERTY 

3.1. Acceptance (1) 

(a) Let the set Y = {xo € R”: f(x) < f (xo)} is bounded. 

(b) Suppose w is a neighbourhood of ¢ then f is continuously differentiable and the Lipchitz condition of 
the gradient is continuous of p. This means, there is k > 0 such that V x. 


lg) — g@)I < klix- £l, € p 


From acceptance (a) and (b) we can design the sequence {x;}e€{ , because f is decreasing. From acceptance 
(a) and (b), we can profit that Y x E€ (4c,,c, > 0 for which ||x|| < c, ||%|| < c> and the sequence {x;}e¢ 
because {f(x;)} is decreasing, henceforward we will assume that assumption (a), (b) are hold and the 
objective function is bounded below. 


Theorem (1): let that acceptance (1) it satisfies, and w holds the Wolfe type line search (8) and (9) and B; is 
given in (26) then (3) holds the property of descent. 


Proof: for (i=1) we get t} = —g, = t7 gı = —|lg:\|* < 0, and this satisfies the descent property. Now we 
have to prove the descent for all k > 1, by multiplying (3) by gi+1: 


thes Gita = —Ilgisill? + Bi t? gira (27) 


if an exact line search is used then tf gj41 = 0 = th, Ji+ı = —Il9i41ll? < 0, but if we used inexact line 
search then (27) yield: 


thea goss = loisa? + [ipe — tas, (o Wirlieytsties) _ 2o )| F gina) (28) 
from (SWC) and (10) and the equality: 

Vi Jira < Ilgis ll? (29) 
and (8), (9) and since: 

ti Yi = ti (Gini - 9) = (S - Dgi ti (30) 


then: 


Igigall? — (p297 td (-6lfi+1-fi-SiJi+1) 
tha Ioa $ -Mgal + [e Eeto fialsa oo) | (—p, gt) 


2 Tt, Tis Tt, 
< Ilg. 24 | llgi+all + (P29; ti) (Seca ti-wp2g; ti) 5 )| a Tt, 
< -|I9i+1Il G-Dal tj try; G-Datt; w || (—P2g; ti) 
2 T52 

< Ilg. I2 — T, lIis _ (p29; ti) (“eee A ) 
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p 
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p2 
-1) 
tla Gis1 SM ||gi+1ll?, which completes the proof. To state the global convergence for the new 
algorithms, you should see [13], [25] which is containing the Zoutindijk condition. 


let k= 


and 0<«<1< is negative, then (1 +k) = m is a positive number then (31) satisfies 


4. THE GLOBAL CONVERGENCE PROPERTY 
Theorem (2): suppose that acceptance (1) holds, consider the algorithm (1) satisfies Wolfe condition, then: 


jim infll gill = 0 (32) 
Proof: If the theorem is not true, then 3 3 > 0 , s.t || gj41|| > 3, Yk, then (3) can be written as: 
Jir + tins = Biti (33) 
by squaring both sides of (33) and rearranging it yields: 
ltil? = = Ilgis? — 2 gi tiga + (Br) Meal? 
= (Bi)? Mtl? = 2 gies tiri = Ilgis ll? (34) 
dividing both sides of (34) by (9/41 tia1)?: 


A 
ltil? BDZ 2 Mgal? 
T i T T T 

(Ji+1 ti+1)? (Ji+1 ti+1)? Giada Cita (Ji+1 tit)? 


_ (BD? Weil? -( lgi 1 ) 1 
(ota tits)? = otis tits lgisll lgi+ll? 
(Bj)? Iltill? 1 
<p t 35 
(gas ti+1)? llgi+1ll? (35) 
now from (26), (29), (30) and the (8)-(10) and that t7 g; = —||t;||? we have: 
^ llgi+1 ll? p29) ti (Seale prole) ) 
a Pagi i (| en ene a 
Bis (s-1) 9} ti + Tita (1-slleqll? an 28) 
lgi+1ll? P29) ti ee ) 
ag Padi tr = (ee g 7 
Scoe Gae aa A Gy) 
llgi+1ll? 
< itt 
~ —(ş-1)llt;l12 (38) 
A ; 2 2 
by squaring (38), we have (fi)? = (25) and sub it in (35): 
Ful a i 
lti+all? < gia? litil? 1 
(atia) O CG-D) NENE (9a tia) Misali? 
1 1 1 1 
< + =—+-=D 39 
ltil? igigall? Da 7 2 (32) 
2 
since ||t l? = -g7 tı = ||g4 ||? by noting that lleil z= n then (39) yields that: 
(go to) litil 
Weiss ll? k 1 1 
(Iira tis)? i51 iigill? D2 Dna 


this contradiction to Zoutendijk condition and with this contradiction, we complete the proof of the theorem. 


5. NUMERICAL FACTS 

The primary goal of this work is to compire and compute the proposed method's execution for a set 
of test functions against the well-known HS routine. These test experiments were collected by Andrei [26]. 
We select (20) large-scale test problems and consider two dimensions (n=100, n=1000) for each test. 
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The stop criterion is ||gi+1 || < 1076, all codes were written in Fortran 90. We denote the number of iterations 
as (NuI) and (NuF), (NuR) as the number of evaluation functions and restarts. All of these results are 
reported in the Table 1 while the percentage performance with respect to (NuD), (NuF) and (NuR) is denoted 
as 92.75%, 72.53%, 61.71% respectively. 


Table 1. Numerical results of a new algorithm 


Functions dim bi HS 
Nul NuR NuF Nul è NuR NuF 
Extended Beale 100 14 8 26 13 7 26 


1000 17 10 32 17 10 32 
100 9 6 25 9 6 25 
1000 22 13 47 61 53 1290 
100 58 21 101 61 19 103 
1000 204 69 351 207 59 339 
100 22 6 44 22 6 44 
1000 27 12 50 27 134 50 
100 7 4 15 7 4 15 
1000 13 7 26 13 7 26 
100 17 9 25 18 10 26 
1000 14 9 25 13 7 24 
100 41 15 61 41 15 61 
1000 52 20 83 62 23 97 


Penalty 

Diagonal 2 

Generalized Tridiagonal 1 
Extended Tridiagonal 1 
Extended Three Expo Terms 


Generalized Tridiagonal 2 


100 å 4 2 8 4 2 8 
Didgonala 1000 4 2 8 4 2 8 
EEEE 100 55 18 103 56 19 1% 


1000 79 22 149 76 26 140 
100 49 10 89 73 18 129 
1000 182 29 324 193 31 343 
100 23 8 46 23 8 46 


Quadratic Diagonal Perturbed 


Extended Wood WOODS 1000 24 10 47 25 10 49 
i 10 6 3 B 6 2. B 
Hifamelbh 1000 6 3 B 6 a. nB 
eo 100 15 8 30 16 8 3l 
1000 11 6 2 R 7 25 

f 10 6 1 B 7 1o 15 
Dqdrtic 1000 7 I is 7 1o 15 
eee 100 10 10 18 10 10 18 
eae 1000 11 11 19 11 11 19 
E 100 17 10 31 17 10 3l 


1000 22 11 47 23 12 52 
100 11 7 21 11 7 21 


Extended Block-Diagonal BD2 1000 12 8 24 12 8 24 


l 100 3 3 9 3 3 9 

Diagonaly 1000 4 4 u 4 3 n 

; f 100 36 14 57 37 13 57 

Generalized quartic GQ2 1000 33 11 53 31 9 55 

BR 100 9 6 17 9 6 17 

eee 1000 9 6 18 9 6 18 
Total 1165 433 2116 1256 597 3429 


6. CONCLUSION 

We have suggest a recent memoryless algorithm depending on scaling the (BFGS) formula. Where this 
newly method produces a sufficient descent direction while this property depends on the type of line search that 
is used in the algorithm which is important. We proposed the global convergence and the numerical results 
which are produced in the previous section showing the percentage of the method efficiency. 
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